Expectation-Maximization via Pretext-Invariant Representations

Expectation Maximization

Expectation Maximization

The Expectation Maximization (EM) algorithm [1, 2] is one of the most widely used algorithms in statistics. Suppose we are given some observed data X and a model family parametrized by θ, and would like to find the θ which maximizes p(X |θ), i.e. the maximum likelihood estimator. The basic idea of EM is actually quite simple: when direct maximization of p(X |θ) is complicated we can augment the...

Week 9: Expectation Maximization

Last week, we saw how we could represent clustering with a probabilistic model. In this model, called a Gaussian mixture model, we model each datapoint x i as originating from some cluster, with a corresponding cluster label y i distributed according to p(y), and the corresponding distribution for that cluster given by a multivariate Gaussian: p(x|y = k) =

Relational Neural Expectation Maximization

Common-sense physical reasoning is an essential ingredient for any intelligent agent operating in the real-world. For example, it can be used to simulate the environment, or to infer the state of parts of the world that are currently unobserved. In order to match real-world conditions this causal knowledge must be learned without access to supervised data. To solve this problem, we present a no...

Neural Expectation Maximization

We introduce a novel framework for clustering that combines generalized EM with neural networks and can be implemented as an end-to-end differentiable recurrent neural network. It learns its statistical model directly from the data and can represent complex non-linear dependencies between inputs. We apply our framework to a perceptual grouping task and empirically verify that it yields the inte...